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1.  SCOPE  AND  USE  OF  TOP  1-1-015. 

1.1  Scope.  This  Test  Operations  Procedure  (TOP)  provides  guidance  for  conducting 
MANPRINT  domain  test  and  analysis  of  all  types  of  materiel  and  systems  tested  by  the 
Developmental  Test  Command  (DTC).  Several  other  MANPRINT -related  TOPs  are  cited  in  this 
TOP.  The  material  presented  in  those  TOPs  augment  the  information  presented  here  and  points 
the  reader  to  other  documents  for  further  guidance. 

TOP  1-1-015  provides  test  procedures,  appropriate  references,  and  general  guidance  for 
conducting  developmental  tests  across  the  seven  MANPRINT  domains.  Domains  include 
Manpower,  Personnel,  Training,  Human  Factors  Engineering,  System  Safety,  Health  Hazards, 
and  Soldier  Survivability. 

This  TOP  also  provides  test  procedure  frameworks  for  domain-specific  testing.  The  scope  of 
MANPRINT  tests  during  the  developmental  test  phases  is  specified  in  DTC  PAM  73-1 1  .  The 
scope  of  MANPRINT  testing  during  customer  tests  will  be  limited  to  that  specified  by  the 
customer. 

The  material  contained  in  this  TOP  does  not  include  detailed  methods  for  statistical  analysis  of 
data.  Guidance  for  statistical  treatment  of  data  is  contained  in  standard  sources  such  as  National 
Institutes  of  Standards  and  Technology’s  E-Handbook  of  Statistical  Methods  (located  at 
http://www.itl.nist.gov/div898/handbook/)  and  Electronic  Textbook:  StatSoft3  (located  at 
http://www.statsoft.com/textbook/stathome.html). 

1.2  MANpower  and  PeRsonnel  INTegration  (MANPRINT)  and  Human  Systems 

Integration  (HSI)  Overview.  HSI  is  a  Department  of  Defense  (DoD)  effort  designed  to 
optimize  the  human  part  of  the  total  system  equation  by  integrating  all  of  its  elements  into  the 
system  acquisition  process.  HSI  is  a  central  theme  for  system  development  throughout  DoD 
directives,  instructions,  and  acquisition  guidebooks,  and  provides  a  common  tenninology  which 
fosters  the  implementation  of  joint  Test  and  Evaluation  (T&E).  It  also  addresses  the  need  for 
human-centered  engineering  throughout  system  design,  development,  and  operation.  HSI 
implementation  is  essential  for  maximizing  total  system  perfonnance  and  reducing  total  lifecycle 
costs. 

Analogous  to  HSI  is  the  MANPRINT  program.  MANPRINT  is  the  Army's  process  to  fulfill 
DoD  HSI  directives  and  includes  seven  domains  which  are  generally  comparable  to  HSI 
elements.  The  Army’s  MANPRINT  Program  is  the  implementation  of  DOD’s  HSI  elements. 

The  term  reflects  current  Army  Regulation  602-24  (Manpower  and  Personnel  Integration  in  the 
System  Acquisition  Process)  terminology.  This  regulation  also  specifies  agencies  responsible  for 
MANPRINT  and  MANPRINT  domain  assessments  conducted  at  major  acquisition  program 
milestone  decisions. 

As  with  HSI  the  MANPRINT  program  ensures  that  Soldier  and  unit  needs  are  considered 
throughout  the  system  acquisition  process  and  life  cycle.  For  clarity  in  this  document,  the  tenn 
HSI  is  understood  to  encompass  both  HSI  elements  and  MANPRINT  domains. 


Superscript  numbers  correspond  to  those  in  Appendix  C,  References 
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Human  Systems  Integration  (HSI)  consists  of  the  following  eight  elements: 

1.2.1  Manpower. 

Manpower  factors  are  those  job  tasks,  operation/maintenance  rates,  associated 
workload,  and  operational  conditions  (e.g.,  risk  of  hostile  fire)  that  are  used  to 
determine  the  number  and  mix  of  military  and  DoD  civilian  manpower  and 
contract  support  necessary  to  operate,  maintain,  support,  and  provide  training  for 
the  system. 

1.2.2  Personnel. 


Personnel  factors  are  those  human  aptitudes  (i.e.,  cognitive,  physical,  and  sensory 
capabilities),  knowledge,  skills,  abilities,  and  experience  levels  that  are  needed  to 
properly  perform  job  tasks.  Personnel  factors  are  used  to  develop  the  military 
occupational  specialties  (or  equivalent  DoD  Component  personnel  system 
classifications)  and  civilian  job  series  of  system  operators,  maintainers,  trainers, 
and  support  personnel. 


1.2.3  Training. 


Training  is  defined  as  the  learning  process  by  which  personnel  individually  or 
collectively  acquires  or  enhances  predetermined  job-relevant  knowledge,  skills, 
and  abilities  by  developing  their  cognitive,  physical,  sensory,  and  team  dynamic 
abilities.  The  "training/instructional  system"  integrates  training  concepts  and 
strategies  and  elements  of  logistic  support  to  satisfy  personnel  performance  levels 
required  to  operate,  maintain,  and  support  the  systems. 


1.2.4  Human  Factors  Engineering. 

Human  factors  are  the  end-user  cognitive,  physical,  sensory,  and  team  dynamic 
abilities  required  to  perform  system  operational,  maintenance,  and  support  job 
tasks.  The  human  factors  engineering  community  integrates  the  human 
characteristics  of  the  user  population  into  the  system  definition,  design, 
development,  and  evaluation  processes  to  optimize  human-machine  performance 
for  both  operation  and  maintenance  of  the  system. 


1.2.5  Systems  Safety. 


Systems  safety  factors  consist  of  those  system  design  characteristics  that  serve  to 
minimize  the  potential  for  mishaps  causing  death  or  injury  to  operators  and 
maintainers  or  threaten  the  survival  and/or  operation  of  the  system.  Prevalent 
issues  include  factors  that  threaten  the  safe  operation  and/or  survival  of  the 
platform;  walking  and  working  surfaces  including  work  at  heights;  pressure 
extremes;  and  control  of  hazardous  energy  releases  such  as  mechanical,  electrical, 
fluids  under  pressure,  ionizing  or  non-ionizing  radiation,  fire,  and  explosions. 
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1.2.6  Health  Hazards. 

Health  hazard  factors  are  those  system  design  features  that  serve  to  minimize  the 
risk  of  injury,  acute  or  chronic  illness,  or  disability;  and/or  reduce  job 
performance  of  personnel  who  operate,  maintain,  or  support  the  system.  The 
design  features  can  include  the  operating  characteristics  of  a  system  that  create 
significant  risks  of  bodily  injury  or  death;  prominent  sources  of  health  hazards 
include:  acoustics  energy,  chemical  substances,  biological  substances, 
temperature  extremes,  radiation  energy,  oxygen  deficiency,  shock  (not  electrical), 
trauma,  and  vibration. 

1.2.7  Soldier  Survivability. 

Personnel  survivability  factors  consist  of  those  system  design  features  that  reduce 
the  risk  of  fratricide,  detection,  and  the  probability  of  being  attacked;  and  that 
enable  the  crew  to  withstand  man-made  hostile  environments  without  aborting  the 
mission  or  suffering  acute  chronic  illness,  disability,  or  death. 

1.2.8  Habitability. 

Habitability  factors  are  those  living  and  working  conditions  that  are  necessary  to 
sustain  the  morale,  safety,  health,  and  comfort  of  the  user  population.  They 
directly  contribute  to  personnel  effectiveness  and  mission  accomplishment,  and 
often  preclude  recruitment  and  retention  problems.  Examples  include:  lighting, 
space,  ventilation,  and  sanitation;  noise  and  temperature  control  (i.e.,  heating  and 
air  conditioning);  religious,  medical,  and  food  services  availability;  and  berthing, 
bathing,  and  personal  hygiene. 

1.3  Using  TOP  1-1-015.  This  TOP  provides  testers  guidance  in  the  development  of  test 
plans  and  execution  of  MANPRINT-related  tests.  It  is  not  intended  to  be  used  as  a  stand  alone 
document  which  fully  addresses  MANPRINT  domains.  Instead,  it  provides  a  general  test 
procedural  framework  which  needs  to  be  supplemented  using  domain-specific  TOPs  and  other 
appropriate  guidance. 

1.4  Applicable  Documents.  There  are  several  regulations/instructions/pamphlets  that 
reference  HSI  test  and  test-related  activities.  These  documents  are  helpful  in  understanding 
testing  requirements,  responsible  agencies,  and  reporting  authorities.  The  key  HSI-related 
documents  are  listed  below: 

1.4.1  DoD  Instruction  5000.2  (12  MAY  03)5:  Operation  of  the  Defense  Acquisition 

System.  This  establishes  a  simplified  and  flexible  management  framework  for  translating 
mission  needs  and  technology  opportunities,  based  on  approved  mission  needs  and  requirements, 
into  stable,  affordable,  and  well-managed  acquisition  programs  that  include  weapon  systems  and 
automated  information  systems  (AISs). 
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1.4.2  Defense  Acquisition  Guidebook6  (16  DEC  04):  Chapter  6,  Human  Systems 
Integration.  This  Guidebook  addresses  the  human  systems  elements  of  the  systems  engineering 
process.  It  will  help  the  program  manager  design  and  develop  systems  that  effectively  and 
affordably  integrate  with  human  capabilities  and  limitations;  and  it  makes  the  program  manager 
aware  of  the  staff  resources  available  to  assist  in  this  endeavor. 

1.4.3  MIL  Handbook  759C7  (31  JUL  95):  Handbook  for  Human  Engineering  Design 
Guidelines.  This  handbook  provides  basic  guidelines  and  data  on  human  engineering  (HE) 
design  for  military  systems,  equipment,  and  facilities.  This  handbook  has  been  designed  to 
supplement  MIL-STD-1472F8. 

1.4.4  MIL  Standard  1472F9  (23  AUG  99):  Human  Engineering.  This  standard  establishes 
general  human  engineering  (HE)  criteria  for  design  and  development  of  military  systems, 
equipment  and  facilities.  Its  purpose  is  to  present  human  engineering  design  criteria,  principles 
and  practices  to  be  applied  in  the  design  of  systems,  equipment  and  facilities  so  as  to  1)  achieve 
required  performance  by  operator,  control  and  maintenance  personnel,  2)  minimize  skill  and 
personnel  requirements  and  training  time,  3)  achieve  required  reliability  of  personnel-equipment 
combinations,  and  4)  foster  design  standardization  within  and  among  systems. 

1.4.5  MIL  Standard  1474D 10  (12  FEB  97):  Noise  Limits.  This  standard  establishes 
acoustical  noise  limits  and  prescribes  testing  requirements  and  measurement  techniques  for 
determining  confonnance  to  the  noise  limits  specified  in  this  standard.  It  is  also  intended  to 
address  noise  levels  emitted  during  the  full  range  of  typical  operational  conditions. 

1.4.6  MIL  Handbook  46855A11  (17  MAY  99):  Human  Engineering  Program  Process  and 
Procedures.  This  handbook  provides  Human  Engineering  (HE)  program  task  guidance, 
describes  the  significance  of  the  analysis,  design,  and  test  aspects  of  the  HE  program,  outlines 
procedures  found  effective  in  implementing  such  guidance,  and  provides  summaries  of  methods. 

1.4.7  Army  Regulation  (AR)  40-1012  (01  OCT  91):  Health  Hazard  Assessment  Program 
in  Support  of  the  Army  Material  Acquisition  Decision  Process.  This  prescribes  specific 
responsibilities  of  developers  for  Health  Hazard  Assessments  (HHAs)  in  support  of  Army 
Acquisition.  Describes  HHA  as  a  cradle-to-grave  integrated  effort 

1.4.8  AR  70-813  (31  JUL  90):  Soldier-Oriented  Research  and  Development  in  Personnel 
and  Training  (SORD-PT).  This  establishes  the  SORD-PT  program,  the  goal  of  which  is  to 
provide  the  technological  edge  necessary  to  ensure  that  the  Army  can  recruit  and  maintain  a  total 
force,  trained  and  ready  to  carry  out  its  assigned  roles  anywhere  in  the  world,  anytime. 
Essentially,  this  covers  the  training  aspects  of  MANPRINT. 

1.4.9  AR  70-2514  (25  JAN  90):  Use  of  Volunteers  as  Subjects  of  Research.  This  regulation 
prescribes  Anny  policy  on  the  conduct  and  management  of  human  subjects  in  testing,  including; 
command  responsibilities,  review  process  requirements,  approval  authorities,  and  reporting 
requirements. 
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1.4.10  AR  70-7515  (02  MAY  05):  Survivability  of  Army  Personnel  and  Materiel.  This 
regulation  implements  survivability  as  outlined  in  Department  of  Defense  Instruction  (DODI) 
5000.2  and  the  Interim  Defense  Acquisition  Guidebook  (Fonnerly  the  DoD  5000. 2-R)  16  within 
the  Army.  It  prescribes  combat  survivability  policies,  responsibilities,  and  procedures  for  the 
sustainment  of  operational  effectiveness  and  warfighting  capability  through  the  life  cycle  of 
survivable  systems,  personnel,  equipment,  and  support.  Combat  survivability  is  the  capability  of 
a  system  to  avoid  (susceptibility)  or  withstand  (vulnerability)  man-made  hostile  environments. 
The  term  “survivability”  includes  both  Soldier  and  equipment  unless  otherwise  specified. 

1.4.11  AR  73-1 17  (01  AUG  06):  Test  and  Evaluation  Policy.  This  regulation  implements  the 
policies  and  procedures  contained  in  Department  of  Defense  Directive  (DODD)  3200. 1 1  , 
DODD  5000. 119,  DODI  5000.2,  and  the  Defense  Acquisition  Guidebook  and  specifically 
prescribes  implementing  policies  for  the  Anny’s  testing  and  evaluation  program. 

1.4.12  AR  350-120  (09  MAY  03):  Army  Training  and  Leader  Development.  This  regulation 
is  a  consolidation  of  several  regulations  that  provide  policy  and  guidance  concerning  Anny 
training  and  education.  This  regulation  updates  policies,  objectives,  and  programs;  aligns 
doctrine  for  the  management  of  Army  training  and  education  programs  in  support  of  a  force- 
projection  Anny;  defines  responsibilities  of  commanders  to  conduct  and  manage  Army  training 
and  education;  and  provides  new  guidance  concerning  the  Army  School  System,  Army 
modernization  training,  and  selected  warfighter,  civilian,  and  leader  training  and  education 
programs. 

1.4.13  AR  385-1021  (29  FEB  00):  The  Army  Safety  Program.  This  prescribes  the 
Department  of  the  Army  (DA)  policy,  responsibilities,  and  procedures  to  protect  and  preserve 
Army  personnel  and  property  against  accidental  loss.  It  provides  for  public  safety  incident  to 
Army  operations  and  activities,  and  safe  and  healthful  workplaces,  procedures,  and  equipment. 
This  regulation  assures  statutory  and  regulatory  compliance. 

1.4.14  AR  385-1622  (02  NOV  01):  Systems  Safety  Engineering  and  Management.  This 
regulation  implements  the  DODI  5000. 3623.  It  updates  Army  system  safety  engineering  and 
management  policy  and  reflects  realignment  of  Army  Staff  safety  responsibilities,  reorganization 
of  the  research,  development,  and  acquisition  process,  risk  management  procedures,  and  updates 
tenninology. 

1.4.15  AR  570-524  (30  JUN  89):  Manpower  Staffing  Standards  System.  This  prescribes  the 
detailed  policies  and  procedures  for  the  development  of  manpower  staffing  standards.  The 
regulation  provides  valuable  orientation  and  guidance  for  those  whose  responsibilities  require 
their  participation  in  the  development  of  manpower  staffing  standards. 

1.4.16  AR  602-123  (08  Feb  91):  Soldier  Materiel  Systems,  Human  Factors  Engineering 
Program.  This  establishes  policies,  procedures,  requirements,  and  assigns  responsibilities  for 
Human  Factors  Engineering  (HFE)  in  the  Department  of  the  Army.  It  emphasizes  front-end 
planning  and  Soldier  performance  database  development. 
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1.4.17  AR  602-226  (01  JUN  01):  Manpower  and  Personnel  Integration  (MANPRINT)  in 

The  System  Acquisition  Process.  This  describes  MANPRINT  and  emphasizes  front-end 
planning  of  soldier-system  design  for  optimum  total  system  perfonnance  as  Army  Acquisition 
Policy.  It  also  explains  the  role  of  the  support  agencies  to  the  MANPRINT  office.  These 
support  agencies  conduct  the  testing  and  reporting  of  MANPRINT  domain  items. 

1.4.18  AR  700-12727  (19  DEC  05):  Integrated  Logistics  Support.  This  prescribes 
Department  of  the  Army  policies  and  assigns  responsibilities  for  the  management  of  life-cycle 
logistics  as  authorized  by  DODD  5000.1  and  DODI  5000.2.  Integrated  logistics  support  (ILS)  is 
the  process  used  by  the  Army  to  implement  these  mandatory  life-cycle  logistics  policies  and 
procedures  and  includes  all  elements  of  planning,  developing,  acquiring,  and  supporting  Army 
materiel  throughout  its  life  cycle. 

1.4.19  Department  of  the  Army  Pamphlet  73-128  (30  MAY  03):  Test  and  Evaluation  in 
Support  of  Systems  Acquisition.  This  provides  guidance  and  procedures  to  implement  test  and 
evaluation  (T&E)  policy  for  materiel  and  infonnation  systems  with  regard  to  planning, 
executing,  and  reporting  T&E  in  support  of  the  acquisition  process  as  promulgated  by  Army 
Regulation  (AR)  73-1“  :  Test  and  Evaluation  Procedures.  Developing  and  deploying  Army 
systems  that  are  operationally  effective,  suitable,  and  survivable  represents  a  significant 
challenge  to  all  involved  in  the  systems  acquisition  process. 

1.4.20  Army  Test  Evaluation  Command  (ATEC)  Pamphlet  73-1 30  (07  SEP  04):  Test  and 
Evaluation  Procedures.  This  pamphlet  provides  background  information  on  integrated  T&E 
strategies  and  provides  guidance  and  suggestions  for  preparing  and  formatting  planning 
documentation  for  tests,  evaluations  and  assessments.  This  process  supports  development  of 
System  Evaluation  Plans  (SEP),  Event  Design  Plans  (EDP),  Detailed  Test  Plans  (DTP),  System 
Evaluation  Reports  (SER),  and  System  Assessments  (SA). 

1.4.21  Developmental  Test  Command  (DTC)  PAM  73-1  (30  OCT  06):  Developmental  Test 
Guide.  This  pamphlet  provides  background  information  on  developmental  testing  strategies  and 
provides  guidance  and  suggestions  for  preparing  and  formatting  planning  documentation  for 
tests,  evaluations  and  assessments.  This  document  describes  the  DTC  MANPRINT  testing 
program  in  Section  9-12. 

1.4.22  DTC  PAM  602-131  (25  July  75):  Questionnaire  and  Interview  Design  (Subjective 
Testing  Techniques).  It  presents  techniques  for  the  design  and  administration  of  questionnaires 
and  interviews,  as  well  as  procedures  for  treatment  of  the  data. 

1.4.23  DTC  Test  Operating  Procedure  (TOP)  1-1-05932  (30  NOV  85):  Soldier-Computer 
Interface.  This  TOP  is  intended  to  be  used  for  the  HFE  Evaluation  of  the  Soldier-Computer 
Interface  of  systems  under  test.  It  encompasses  procedures  for  an  HFE  analysis  and  walk¬ 
through,  mission  simulation,  and  interview  guide. 

1.4.24  DTC  TOP  1-1-06033  (07  APR  86):  System  Safety  Engineering.  This  TOP  provides 
guidance  for  identifying  and  evaluating  hazards  associated  with  systems  under  test.  The  purpose 
is  to  provide  uniform  requirements  and  criteria  for  performing  system  safety  analysis  during  the 
course  of  testing  material.  (This  TOP  has  been  superseded  by  ITOP  5-1-060  '4.) 
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1.4.25  DTC  TOP  1-2-608'5  (17  JUL  81):  Sound  Level  Measurements.  This  TOP  describes 
procedures  for  measuring  the  sound  levels  of  system  under  test.  It  covers  tests  for  steady-state 
noise  from  military  vehicles  and  general  equipment,  and  impulse  noise  from  weapons  systems 
and  explosive  ordnance  materiel. 

1.4.26  DTC  TOP  1-2-61036  (15  MAY  90):  Human  Factors  Engineering.  This  encompasses 
the  HFE  procedures  for  the  testing  of  design,  functional  performance  and  environmental 
considerations  for  the  major  test  functions  (operability,  maintainability,  transportability, 
portability,  usability,  erectability,  and  habitability)  applicable  to  the  HFE  assessment. 

1.4.27  DTC  TOP  2-2-508  7  (24  NOV  82):  Automotive  Safety  and  Health  Hazard 
Evaluation.  This  TOP  provides  procedures  to  identify  and  evaluate  real  and  potential  safety  and 
health  hazards  that  exist  in  military  tracked  and  wheeled  vehicles. 

1.4.28  DTC  TOP  2-2-80838  (01  OCT  81):  Field  Shock  and  Vibration  Test  of  Vehicles.  This 
TOP  provides  a  method  for  evaluating  shock  and  vibration  characteristics  of  vehicles  during 
operation  over  test  courses.  Describes  instrumentation  and  courses,  and  provides  guidelines  for 
determining  points  at  which  three  standardized  levels  of  human  exposure  are  reached. 

1.4.29  DTC  TOP  3-2-50339  (15  AUG  80):  Safety  Evaluation  of  Fire  Control  Systems  - 
Electrical  and  Electronic  Equipment.  This  TOP  provides  procedures  for  evaluating  the  safety 
of  electrical  and  electronic  equipment  in  fire  control  systems  for  tank  weapons  and  field  and  air 
defense  artillery.  Checklists  are  included  as  a  guide  for  identifying  electrical  and  electronic 
hazards,  mechanical  hazards  and  miscellaneous  hazards. 

1.4.30  DTC  TOP  3-2-50440  (01  MAR  77):  Safety  Evaluation  of  Hand  and  Shoulder 
Weapons.  This  TOP  provides  procedures  for  evaluating  the  safety  of  hand  and  shoulder 
weapons. 

1.4.31  DTC  TOP  3-2-71141  (02  DEC  82):  Safety  Evaluation  -  Radioactive  Components  of 
Materiel.  This  TOP  describes  procedures  for  evaluating  the  radiological  safety  of  materiel 
components  that  emit  ionizing  radiation.  Contents  include  shock,  vibration,  and  climatic  tests 
and  a  test  to  detennine  whether  combined  storage  will  result  in  radiation  or  contamination. 

1.4.32  DTC  TOP  3-2-81242  (23  FEB  66):  Field  of  Vision  -  Vehicles.  This  TOP  describes 
procedures  to  determine  the  field  of  vision  for  the  driver  of  transport  vehicles  and  for  the  entire 
crew  of  combat  vehicles. 

1.4.33  DTC  TOP  3-2-81343  (23  MAR  85):  Field  of  Fire.  This  TOP  provides  procedures  for 
determining  the  field  of  fire  for  vehicle-mounted  primary  and  secondary  armament  (e.g.,  tank 
guns,  armored  personnel  carriers). 

1.4.34  DTC  TOP  4-2-50244  (05  MAY  78):  Safety  Evaluation  of  Mines  and  Demolitions. 

This  TOP  provides  procedures  for  evaluating  the  safety  of  mines  and  demolitions.  It  covers 
inspections  and  tests  for  adequacy  of  safety  features;  confirmation  of  functioning  loads; 
sensitivity  to  accidental  detonation;  safety  during  transportation. 
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1.4.35  DTC  TOP  4-2-50345  (01  July  70):  Safety  Evaluation  -  Close  Support  Rockets  and 
Missies.  This  TOP  provides  guidance  for  determining  whether  close  support  rockets  and 
missiles  are  safe  for  testing. 

1.4.36  DTC  TOP  4-2-50446  (01  APR  79):  Safety  Testing  of  Artillery,  Mortar,  and 
Recoilless  Rifle  Ammunition.  This  TOP  describes  safety  evaluation  test  procedures  applicable 
to  all  ammunition  for  field  and  air  defense  artillery,  tank  guns,  recoilless  rifles,  and  mortars. 

1.4.37  DTC  TOP  6-2-50747  (15  JUN  81):  Safety  and  Health  Evaluation  of  Hand  - 
Communication/Electronic  Equipment.  This  TOP  provides  procedures  for  evaluating 
potential  hazards  during  testing.  Safety  checklists,  physical  tests,  observations,  and 
examinations  are  presented  for  use. 

1.4.38  DTC  TOP  7-3-52648  (05  FEB  93):  External  Acoustical  Noise  Measurements  for 
Aviation  Systems.  This  TOP  provides  procedures  for  measuring  the  external  acoustical  noise  of 
Army  helicopters.  It  covers  test  procedures  for  the  measurements  of  steady-state  and  impulse 
noise. 

1.4.39  DTC  TOP  7-3-52949  (30  SEP  91):  Ingress,  Emergency  Egress,  and  Emergency 
Evacuation  Testing  of  Army  Aircraft.  This  TOP  describes  procedures  for  the  ingress, 
emergency  egress,  and  emergency  evacuation  testing  of  army  aircraft.  Also  contains  guidelines 
for  maximum  times  allowed  for  emergency  egress  and  emergency  evacuation. 

1.4.40  DTC  TOP  7-3-530  0  (05  FEB  93):  Steady-State  Acoustical  Noise  Measurements  in 
Aviation  Systems.  This  TOP  provides  procedures  for  measuring  the  acoustical  noise  in  Army 
helicopters.  It  covers  test  procedures  for  the  measurements  of  steady-state  acoustical  noise  at 
crewstations  and  in  the  passenger  compartment. 

1.4.41  DTC  TOP  8-2-11151  (24  April  98):  Nuclear,  Biological,  and  Chemical  (NBC) 
Contamination  Survivability,  Small  Items  of  Equipment.  This  TOP  provides  procedures  for 
facilitating  test  planning,  conducting,  and  reporting.  It  is  to  be  used  primarily  for  testing  of  small 
items  of  equipment  that  are  decontaminated  by  the  individual  Soldier  or  by  small  teams 
operating  portable  and  hand-held  decontaminating  devices. 

1.4.42  DTC  TOP  8-2-55352  (01  AUG  79):  Safety  Evaluation  -  Chemical  Biological  (CB) 
Items.  This  TOP  provides  procedures  for  identifying  and  evaluating  hazards  associated  with  CB 
equipment.  In  addition,  requirements  for  chemical  and  biological  safe  test  conduct  are  outlined. 

1.4.43  DTC  TOP  10-2-20653  (23  OCT  84):  Body  Armor.  This  TOP  provides  test  methods  and 
techniques  for  evaluating  the  technical  perfonnance  and  characteristics  of  body  armor  and 
determining  its  suitability  to  be  subjected  to  further  testing. 

1.4.44  DTC  TOP  10-2-50654  (06  JAN  75):  Ballistic  Testing  of  Personnel  Armor  Materials 
Armor.  This  TOP  describes  methods  of  evaluating  the  resistance  of  the  materiel  used  in 
personnel  annor  to  penetration  by  projectile  fragments,  simulated  fragments,  and  small  arms 
ammunition. 
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1.4.45  DTC  TOP  10-2-50855  (06  MAY  80):  Safety  and  Health  Hazard  Evaluation  - 
General  Equipment  This  TOP  provides  procedures  required  to  determine  whether  general 
equipment  is  free  from  design,  operational  or  maintenance  hazards.  Checklists  and  a  hazard 
analysis  format  are  provided  to  assist  test  personnel  in  the  assessment  of  hazards. 

1.4.46  ITOP  5-1-06034  (10  JUN  05):  Identification  of  Hazards  for  Operators  and 
Maintainers  of  Surface  Launched  Weapons  This  ITOP  is  a  modified  version  of  DTC  TOP  1- 
1-060,  System  Safety  Engineering,  which  is  listed  in  paragraph  1.4.24.  ITOP  5-1-060  contains  a 
safety  checklist  entitled  CECOM  Supplement  1  to  AR  385-16.  This  safety  checklist  addresses 
all  the  provisions  of  Guideline  1  of  MIL-HDBK-45436. 

2.  FACILITIES  AND  INSTRUMENTATION. 

Specific  requirements  for  the  facilities  and  instrumentation  are  included  in  the  specific  test 
procedures  of  the  respective  TOPs  under  this  HSI  topic. 

2.1  Facilities.  Facilities  needed  depend  on  the  classification  of  the  item  under  test  and  the 
required  tests.  The  test  conditions  may  also  drive  the  needed  facilities  for  the  test.  Facilities 
normally  provided  for  other  subtests  are  often  adequate  for  the  requirements  of  HSI  testing.  The 
HSI  tests  that  are  dependent  on  certain  facilities  have  been  described  in  this  TOP  (section  3.10). 

2.2  Instrumentation.  Requirements  generated  in  HSI  test  plans  drive  specific  types  of 
instrumentation.  Choosing  the  right  instrumentation  depends  on  test  requirements, 
instrumentation/facility  availability,  and  funds  available.  Several  examples  of  instrumentation 
are  described  below: 

2.2.1  Human  Performance  Instrumentation.  Instrumentation  for  measuring  human 
performance  may  include  stopwatches  (for  time  task  completion),  video  cameras  (for  recording 
task  completion),  and  questionnaires/surveys/ After  Action  Reviews. 

Test  methodologies  adjust,  as  necessary,  to  new  and  emerging  technologies.  One  area  of  greatest 
change  is  in  human  perfonnance  measurement  and  analysis.  For  example,  tools  addressing 
situational  awareness,  cognitive  and  physical  workload,  and  team  performance  are  changing  how 
items  are  tested  and  the  nature  of  data  collected.  Instrumentation  to  measure  these  areas  is  being 
developed  for  application  to  testing. 

2.2.2  HFE  Instrumentation.  Instrumentation  covering  HFE  testing  include,  for  example, 
items  to  measure:  anthropometry,  atmosphere/environment,  force/torque/dimension, 
illumination,  and  noise.  These  items  are  covered  in  TOP  1-2-610,  Human  Factors  Engineering. 
Some  instrumentation  not  covered  in  TOP  1-2-610,  include  typical  models  and  simulations  used 
in  HFE  testing,  such  as  Jack  and  other  anthropometry  modeling. 

2.2.3  Soldier  Survivability.  Instrumentation  used  in  Soldier  survivability  testing  can  include 
human  surrogates  (dummies  and  mannequins).  Instrumentation  can  also  be  ballistic  impact 
measurement  instrumentation  and  high  speed  cameras  to  record  impacts. 

2.2.4  System  Safety  and  Health  Hazards.  Instrumentation  used  in  system  safety  and  health 
hazard  assessments  is  used  to  collect  MANPRINT  data  to  support  safety  releases,  safety 
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confirmations,  and  Human  Use  Committee  (HUC)  actions.  Instrumentation  can  include  toxic 
gas  analyzers  and  accelerometers  (for  vibration  determination).  Instrumentation  to  measure 
mechanical,  electrical,  chemical,  fire  and  explosive  hazards  can  be  used.  Other  instrumentation 
is  used,  as  required. 

2.2.6  Modeling  and  Simulation.  M&S  has  become  increasingly  important  in  testing. 
Simulations  allow  testers  to  conduct  multiple  runs  very  quickly  at  low  cost.  M&S  is  used  in 
every  domain  of  HSI  testing.  Examples  of  this  can  be  manpower  and  personnel  testing  using  the 
Improved  Perfonnance  Research  Integration  Tool  (IMPRINT)  and  human  performance  testing 
with  the  Infantry  Warrior  Simulation  (IWARS). 

2.2.7  Questionnaires/Surveys.  These  instruments  are  normally  developed  and  administered  to 
obtain  data  from  test  participants.  Questionnaires  and  surveys  can  be  administered  either 
electronically  or  on  paper.  Electronic  versions  decrease  the  need  to  enter  answers  into  databases 
by  hand  and  eliminate  any  ambiguity  from  reading  test  participant’s  handwriting.  Examples  of 
questionnaires/surveys  used  can  be  NASA  TLX  (workload),  Situation  Awareness  Rating 
Technique  (SART),  and  fit/comfort  (equipment  compatibility). 

2.2.8  Other  Domain-specific  Instrumentation.  Other  types  of  instrumentation  can  be  used 
for  the  domain  tests.  Instrumentation  is  not  limited  to  that  listed  in  this  TOP. 

3.  TEST  PREPARATION  METHOD. 


Before  using  this  TOP  to  plan  an  HSI  test,  obtain  the  following  documents  for  the  test  items. 

a.  System  Evaluation  Plan  (SEP) 

b.  Event  Design  Plan 

c.  MANPRINT  Management  Plan 

d.  Test  and  Evaluation  Master  plan  (TEMP) 

e.  Detailed  Test  Plan 

f.  Initial  Capabilities  Document  (ICD)  /  Capability  Development  Document  (CDD) 

g.  Safety  Documentation  &  safety  releases 

h.  Technical  Manuals 

i.  Other  relevant  documents 

These  documents  can  be  obtained  from  the  following  sources: 

a.  The  project  evaluator  from  Army  Evaluation  Center 

b.  The  Program  Manager  at  the  Program  Manager’s  Office. 
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c.  The  Test  Director  at  the  Test  Center 

d.  The  Test  Manager  at  Developmental  Test  Command 

e.  The  Material  Developer 

A  primary  overall  emphasis  in  HSI  testing  is  to  measure  and  analyze  the  soldier’s  impact  on  total 
system  performance  and  the  system’s  impacts  on  Soldier  performance.  Since  the  Soldier  is  an 
integral  component  of  the  system,  total  system  performance  must  be  addressed  to  determine  the 
soldier’s  effect  on  the  system  and  visa  versa.  The  major  requirements  in  preparation  for  an  HSI 
test  are  to  identify  the  item  to  be  tested,  how  the  test  is  to  be  conducted,  and  what  are  the 
evaluation  criteria.  In  determining  what  is  to  be  tested,  the  critical  issues  and  objectives  must 
also  be  detennined  using  the  data  source  matrix  attached  to  the  SEP  and  the  TEMP.  Past  test 
events  and  test  reports  can  ascertain  prior  HSI  concerns.  Objectives,  as  well  as  critical  issues, 
may  be  designated/described  in  system  evaluation  plans  or  other  pertinent  information  related  to 
the  item  under  test.  For  some  programs,  an  ARL-HRED  MANPRINT  POC  maybe  be  assigned 
to  support  the  program.  He/she  will  be  able  to  assist  in  obtaining  program-related  information. 


3.1  Step  1  -  Classify  the  Test  Item.  One  of  the  initial  steps  in  planning  an  HSI  test  is  to 
classify  the  test  item  into  one  or  more  appropriate  test  item  classes.  The  following  are  general 
classes  that  can  be  used  to  group  items  and  detennine  which  types  of  testing  that  will  be 
necessary.  This  type  of  information  may  already  be  found  in  documentation  concerning  the 
item. 


a.  Vehicles  -  Air,  Ground,  Unmanned  and  Manned 

b.  Weapons/Ammunition  -  Individual,  Crew  Served,  or  Ammunition 

c.  Material  Handlers  -  Soldier-Operated  or  Soldier-Monitored 

d.  Electronics/Signals  -  Sensors  &  Detectors  or  Infonnation/Command  &  Control 
Systems 

e.  Operational  Support  -  Maintenance  &  Repair  equipment,  Material  Production  & 
Environmental  Control,  or  Major  Construction  Items 

f.  Troop  Support  -  Consumables,  Clothing  &  Personal  Equipment,  or  Living  & 
Working  Areas 

3.2  Step  2  -  Determine  Applicable  Test  Item  Functions.  Primary  areas  of  concern  in  an 
HSI  test  are  listed  below: 


a.  Operation  of  the  item  in  representative  environments 

b.  Maintenance  of  the  item 

c.  Erectability  of  the  Structure 
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d.  Portability/usability/habitability  of  the  item 

e.  Habitability  of  the  living  conditions  within  the  item 

f.  Personnel  Survivability  of  the  item 

The  test  functions  become  important  as  test  plans  are  written.  The  test  item  functions  describe 
the  operational  enviromnent  the  test  will  be  conducted  in  and  which  procedures  will  be  examined 
during  the  test.  The  documentation  (mainly  ICD/CDD)  will  list  capabilities  and  requirements  of 
the  item  under  test.  The  System  Evaluation  Plan  provides  an  additional  source  for  detennining 
which  functions  should  be  examined  for  testing. 

3.3  Step  3  -  Identify  General  Test  Conditions.  Use  conditions  are  test  user, 
facilities/instrumentation,  environment,  and  operational  variables  that  must  be  controlled  within 
the  test.  These  use  conditions  can  affect  operability,  maintainability,  as  well  as  the  other  test 
functions.  By  controlling  the  use  conditions  within  the  context  of  test  events,  the  variability  (or 
biases)  of  the  item  perfonnance  data  can  be  decreased. 

The  HSI  specialist  planning  the  test  will  ensure  that  the  HSI-related  tests  are  performed  under 
use  conditions  representing  real  world  conditions  to  the  degree  feasible.  These  use  conditions 
can  affect  performance  during  any  of  the  test  item  functions. 

TOPs,  (such  as:  TOP  1-1-00357  Arctic  Personnel  Effects,  and  TOP  1-1-00  658  Desert 
Environment  Considerations)  prescribe  specific  use  conditions  for  certain  type  of  tests.  The 
following  sections  outline  five  categories  of  use  conditions  applicable  to  HSI  testing. 

3.3.1  User  (Test  Participant)  Conditions.  Use  conditions  associated  with  the  test  participant 
(or  intended  item  user)  include  the  individual  differences  between  the  test  participants  and  the 
intended  user  population.  These  individual  differences  can  affect  perfonnance  in  each  of  the  test 
item  functions  and  can  be  identified,  characterized,  and  collected  during  testing  through 
anthropometric  measurements  and  demographic  surveys.  The  following  is  a  list  of  several  areas 
of  individual  test  participant  differences  generally  addressed  during  testing.  Other  items  can  be 
addressed  based  on  program  requirements. 

a.  Gender  (male,  female) 

b.  Anthropometries  (height,  weight,  limb  length,  reach  distance,  sitting  height,  etc.) 

c.  Clothing  worn  (size,  type) 

d.  Personnel  encumbrances  (combat  pack,  weapon,  radio,  etc.) 

e.  Skills  and  knowledge  (MOS,  experience,  training,  situational  awareness,  decision¬ 
making  ability,  etc.) 

f.  Special  considerations  (handedness,  physical  strength,  eye  correction,  facility  of 
English  language,  etc.) 
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3.3.2  Team  Dynamics  Conditions.  As  distributed  testing  capabilities  increase,  measurement 
and  analysis  of  team  performance  becomes  increasingly  important.  How  well  a  team  performs  is 
a  function  of  the  team  dynamics.  By  identifying  team  characteristics  and  controlling  for  them, 
team  characteristics  can  be  used  in  data  analysis  to  decrease  human  variability  within  and 
between  groups.  The  following  is  a  list  of  some  of  the  team  characteristics  that  should  be 
identified  and  measured. 

a.  Unit  Cohesion 

b.  Shared  Situational  Awareness 

c.  Leadership 

d.  Decision-making  abilities  (presence  of  decision-making  aids) 

3.3.3  Facilities/Instrumentation  Conditions.  As  distributed  testing  increases,  documentation 
of  facilities  and  instrumentation  becomes  important  in  test  planning  and  reporting.  Using  one 
facility  and  instrumentation  set  over  another  may  produce  inconsistent  data  across  test  centers  or 
tests  based  on  the  facility/instrumentation.  For  example  consider  one  Military  Operations  in 
Urban  Terrain  (MOUT)  site  over  another.  One  might  use  realistic  buildings,  furniture/debris, 
and  civilians  in  the  scenes,  while  the  second  might  use  a  plywood  shoot  house  without  obstacles. 
Both  will  give  different  perfonnance  times  and  achieve  different  results  which  need  to  be 
addressed  in  data  reduction  and  analysis. 

3.3.4  Environmental  Conditions.  Use  conditions  associated  with  the  environment  include 
climate,  weather,  and  work  conditions  under  which  the  item  will  be  operated  and  the  factors 
associated  with  the  intended  mission  which  affects  the  operability/maintainability  of  the  test 
item.  Listed  below  are  several  environmental  conditions  that  could  affect  performance. 

a.  Precipitation  (rain,  snow,  fog  and  similar  meteorological  conditions  which  affect 
visibility  and  mobility) 

b.  Temperature  (extremes  of  heat  and  cold) 

c.  Humidity  (extremes  -  assessed  in  conjunction  with  temperature) 

d.  Wind  (effects  on  mobility,  handling  of  components) 

e.  Terrain  (type,  evenness,  hardness,  effects  on  mobility  or  footing) 

f.  Ventilation  (effects  on  comfort,  safety  and  performance) 

g.  Airborne  contaminants  (dust,  noxious  fumes) 

h.  Lighting  (type,  location,  levels  -  effects  on  visibility) 

i.  Noise  (spectrum,  loudness,  effects  on  comfort,  safety,  reception  of 
communication,  performance) 
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j.  Vibration  (spectrum  and  intensity,  effects  on  comfort,  safety  and  performance) 

HSI  test  planners  need  to  examine  the  use  environment  for  the  item  under  test.  If  the  test  item  is 
a  computer  workstation  in  a  command  and  control  vehicle,  there  is  a  different  environmental 
condition  than  if  it  is  aboard  a  ship  or  at  a  stationary  command  headquarters. 

3.3.5  Operational  Conditions.  Operational  use  conditions  include  characteristics  of  the 
deployment  and  mission  which  could  affect  operability/maintainability  of  the  test  item.  These 
conditions  must  be  identified  to  ensure  that  the  test  has  operational  realism.  They  will  be 
tailored  to  the  Program  Requirements  as  identified  in  such  documentation  as  the  ICD/CDD 
and/or  TEMP.  These  conditions  include: 

a.  Threat  characteristics  (type,  number,  distance,  deployment) 

b.  Force  characteristics  (mission  profile,  operational  mode  summary,  crew 

composition) 

c.  Conditions  of  readiness 

d.  Blackout  conditions 

e.  Logistical  constraints 

f.  Emergency  conditions 

3.4  Step  4  -  Identify/Analyze  Operator/Maintainer  Tasks.  One  of  the  primary  objectives 
of  developmental  testing  is  to  verify  that  design  characteristics  of  the  test  item  are  or  are  not 
adequate  to  support  the  intended  use  of  the  item.  The  intended  use  is  best  specified  by  a  set  of 
tasks  which  operators  and  maintainers  perfonn.  Once  a  set  of  tasks  has  been  defined  for  the 
applicable  test  functions,  task  checklists  are  created,  by  systemically  going  through  the  tasks  on 
the  checklist. 

3.4.1  Task  Identification  and  Analysis.  Identification  of  tasks  to  be  analyzed  begins  with  a 
review  of  test  item  documentation.  Additional  information  can  be  gathered  from; 

a.  Contractor  Task  Analyses  -  collected  from  the  PM 

b.  Generic  Soldier/Item  Tasks  -  extracted  from  the  Soldier’s  Manual  of  Common 
Tasks 

c.  Legacy  Item  task  lists  -  collected  from  old  test  reports  and  evaluations 

d.  Test  Item  Operating  Manuals  and  Maintenance  Manuals  -  collected  from  the  PM 

e.  Operator/Maintainer  Experience  -  gathered  from  SMEs 

3.4.2  Tailored  Task  Checklists.  After  the  criticality  analysis  and  the  task  sequence  is 
determined,  a  specific  Task  Checklist  can  be  created  appropriate  for  the  item.  The  Task 
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Checklist  must  be  completed  at  this  point  in  the  planning  process  so  that  it  can  be  used  to 
identify  potential  problems  and  areas  where  further  investigation  will  be  required  during  HSI 
testing. 


3.5  Step  5  -  Preliminary  HSI  Analysis.  The  MANPRINT  specialist  or  test  planner  will  use 
any  or  all  available  information,  including  the  technical  data  package,  manuals  and  the  item 
itself,  if  it  is  available,  to  conduct  the  preliminary  HSI  analysis. 

3.5.1  Prepare  Preliminary  Analysis.  The  identified  tasks  from  the  task  checklists  should 
receive  extra  attention  during  the  HSI  test.  For  each  task  identified,  factors  that  affect 
completion  of  the  task  should  be  considered.  The  factors  will  be  the  items  during  the  HSI  test 
that  should  be  studied.  Factors  can  be; 

a.  Environmental  conditions  (e.g.,  effects  of  ambient  noise  on  a  communication  task, 
or  effects  of  illumination  on  an  inspection  task) 

b.  Equipment  characteristics  (e.g.,  size,  shape,  location,  visibility,  etc.,  of 
components) 

c.  Test  participants'  characteristics  (e.g.,  limb  length,  clothing  conditions,  training, 
MOS,  group  dynamics,  and  experience  with  similar  items) 

d.  Perfonnance  (weapon  delivery,  tracking,  road  following,  decision-making,  etc.) 

3.5.2  Walk-through  and  Talk-through.  This  form  of  preliminary  analysis  should  be 
conducted  if  sufficient  information  exists  for  the  operator/maintainer  tasks.  It  can  also  be 
performed  if  the  item  is  available  for  analysis.  An  experienced  operator/maintainer  can  assist  the 
HSI  specialist  in  the  preliminary  analysis.  The  analysis  consists  of  considering  each  task  in 
sufficient  detail  to  identify: 

a.  Potential  HSI  problems  in:  equipment  design  characteristics  (including  system 
safety,  health  hazards,  personnel  survivability,  ergonomics),  environmental 
effects,  and  system  perfonnance 

b.  Test  participant  characteristics  (body  dimensions,  skills,  physical  condition, 
decision-making,  situational  awareness,  state  of  alertness,  stress,  clothing)  to  be 
evaluated 

c.  Types  of  HSI  tests  to  be  included  and  measurements  required 

d.  General  data  requirements 

e.  Facilities  and  instrumentation  required 

3.5.3  Modeling  &  Simulations.  This  form  of  testing  (if  available)  should  be  conducted,  as 
appropriate.  Modeling  &  Simulation  (M&S)  can  take  many  forms  from  physical  mockups  to 
3  Dimensional  renderings  of  the  item.  If  the  item  is  not  immediately  available  for  preliminary 
analysis,  then  a  model  can  be  used  to  approximate  the  final  layout  and  placement  of  the  details 
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on  the  item.  Simulations  can  be  used  to  analyze  the  human’s  interaction  with  the  item.  Human 
figure  models  can  be  run  in  simulation  with  item  models  to  analyze  interactions.  M&S  tools  can 
be  used  in  all  domains  of  HSI  testing  as  they  can  supplement  demonstrations  for  the  same 
purposes. 

3.5.4  Documentation  Review.  Another  type  of  preliminary  analysis  can  be  completed  by 
conducting  a  system  document  review.  By  examining  the  current  program  documents,  legacy 
system’s  documents  and  lessons  learned  the  tester  may  be  able  to  detennine  which  aspects  of  the 
item  need  to  be  studied  to  detennine  the  overall  system  perfonnance. 

3.6  Step  6  -  Identify  Data  Elements.  At  this  point  in  the  test  planning  process,  the 
determination  of  which  data  elements  to  be  collected  must  be  made.  By  identifying  the 
individual  data  elements  to  be  collected,  the  test  procedures  to  collect  those  data  elements  can  be 
prescribed. 

3.6.1  Determine  the  Data  Requirements.  Review  of  the  system  documentation,  especially 
the  system  evaluation  planning  documents  is  required  at  this  stage.  By  reviewing  the  data 
requirements,  the  data  elements  that  need  to  be  collected  can  be  gathered.  Data  requirements  can 
also  be  contained  within  the  ICD/CDD,  TEMP  and  the  SEP. 

3.6.2  Determine  the  Applicable  Test  Item  Components.  The  HSI  test  shall  include 
procedures  for  the  test  and  analysis  of  the  HSI  design,  technical  and  perfonnance  characteristics 
of  the  system.  To  determine  the  specific  characteristics  applicable  to  the  system  being  tested,  the 
HSI  specialist  must  identify  the  Test  Item  Components  with  which  the  operator/maintainer 
interacts  in  performing  the  specified  tasks. 

3.6.3  Determine  Data  Element  and  Item  Function  Pairing.  The  applicable  item  functions, 
identified  in  Section  3.2,  are  used  with  the  required  data  elements  to  determine  test  procedures. 
The  data  elements  that  need  to  be  collected  must  be  matched  with  applicable  test  functions  to 
make  sure  all  data  elements  will  be  collected  with  the  test  procedures  created  during  the  next 
section.  That  is  usually  part  of  the  Detailed  Test  Plan  process. 

3.7  Step  7  -  Select  Test  Procedures.  The  next  step  in  the  process  is  to  decide  on  which  test 
procedures,  methods,  and/or  instrumentation  to  be  used  to  test  and  analyze  the  test  item.  The 
following  is  a  list  of  various  types  of  procedures  that  can  be  used  for  an  HSI  test: 

3.7.1  Equipment,  Technical,  &  Design  Characteristics.  Technical  characteristics  include  the 
mechanical,  thermal,  atmospheric  or  illumination  environments  created  by  the  equipment  during 
use,  and  the  environmental  conditions  under  which  the  equipment  must  be  operated  and 
maintained  (weather,  climate,  temperature,  noise,  terrain,  illumination,  etc.).  Technical 
characteristics  to  be  measured  can  be  determined  from  equipment  documentation.  They  will  be 
assessed  using  specific  test  procedures  extracted  from  other  HSI-related  TOPs. 

Design  characteristics  include  the  physical  dimensions  of  the  equipment  components  which  are 
used  by  or  which  contact  the  equipment  operator/maintainer.  Design  can  also  include  the 
displays  for  a  piece  of  equipment;  as  well  as  the  software  and  how  that  is  displayed  to  the  user. 
Design  characteristics  are  assessed  using  specific  Test  Procedures  from  other  HSI  TOPs. 
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Various  checklists  have  been  designed  to  help  testers  obtain  data.  A  checklist  contains  test 
relevant  items  that  must  be  systematically  addressed.  While  checking  off  an  item,  the  tester 
might  have  to  perfonn  an  action  to  complete  that  item.  For  example,  if  the  item  assesses  vehicle 
entry/egress,  the  tester  may  have  to  measure  the  size  of  the  opening,  the  force  needed  to  open  the 
door,  and  handhold  and  footrest  accommodation  inside  and  outside  the  vehicle.  Several 
measurements  may  be  required  before  checking  a  single  item  on  a  checklist.  Several  types  of 
checklists  are  explained  below. 

3.7.1. 1  Equipment,  Technical  &  Design  Checklists.  These  checklists  can  be  made  for 
assessing  any  type  of  equipment.  Several  different  types  of  checklists  are  explained  below: 

a.  Human  Factors  Engineering/Design  checklists  -  These  checklists  focus  on  design 
requirements  and  specifications  for  that  piece  of  equipment.  HFE  design 
requirements  are  included  in  MIL  STD  1472F  and  noise  requirements  are 
included  in  MIL  STD  147459.  Several  other  MIL  STDs  and  the  equipment’s 
documentation  will  specify  which  requirements  need  to  be  accessed  and 
measured.  These  should  be  consulted  before  any  testing  begins. 

b.  Environment  checklists  include  many  of  the  same  design  requirements  from  HFE 
checklists.  These  items  include  noise,  lighting,  air  pressure,  and  hazards  caused 
by  the  equipment  while  it  is  operating.  Examples  can  be  the  interior  lighting 
inside  a  vehicle,  noise  inside  a  helicopter  or  maintenance  hazards  for  engine 
repair. 

c.  Workspace  accommodations  checklist  assess  the  immediate  environment 
surrounding  the  user.  Items  can  include  workstation  ergonomics,  toxic  fumes, 
head  clearance,  and  sufficient  heating  and  cooling  for  occupants. 

3.7.1.2  Safety  Review.  This  type  of  checklist  deals  with  items  that  may  affect  the  safety  of  the 
Soldier  while  he/she  operates/maintains  the  piece  of  equipment.  Items  to  examine  include 
mechanical  hazards,  electrical  hazards,  and  personnel  safeguards,  (e.g.  sufficient  lighting  to 
work,  system  shutdowns  and  lockout  procedures  for  maintainers,  and  a  habitable  work 
environment  for  occupants) 

3.7.1.3  Health  Hazard  Checklist.  This  type  of  checklist  deals  with  items  that  may  affect  the 
long-term  safety/well  being  as  well  as  short-tenn  perfonnance  of  the  operator/maintainer.  Items 
can  include  toxic  fumes,  excessive  vibration,  temperature  extremes,  and  radiation  levels. 

Specific  industrial  hygiene  measurements  should  be  made  to  verily  suspected  hazards. 

3.7.1.4  Survivability  Checklist.  This  type  of  checklist  deals  with  items  that  directly  affect  the 
survivability  of  the  Soldier  during  operation.  Items  such  as  ballistic  protection,  decreasing 
fratricide,  and  maintaining  air  pressure  all  affect  the  survivability  of  the  Soldier  during  operation 
of  a  piece  of  equipment. 

3.7.2  System  Performance.  The  evaluation  of  human  integration  as  part  of  the  total  system 
performance  is  important.  How  the  user  interacts  with  the  equipment  can  impact  greatly  on  the 
performance  of  the  total  system.  While  the  user  affects  the  performance  of  the  equipment,  the 
equipment  affects  the  performance  of  the  user.  This  symbiotic  relationship  is  difficult  to 
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measure  but  essential  for  measuring  total  system  perfonnance.  Perfonnance  of  the  user  can  be 
measured  at  three  different  levels:  individual,  team/crew,  and  distributed. 

a.  Individual  performance  deals  with  measures  concerning  the  individual 
operator/maintainer.  Items  such  as  individual  task  completion  time,  individual 
situational  awareness,  cognitive  workload  and  error  rate  can  be  collected 
concerning  only  one  individual.  These  measures  all  impact  the  individual 
performance  as  it  pertains  to  mission  success. 

b.  Team/crew  performance  measures  concentrate  on  team  aspects  such  as  team 
Situational  Awareness  (SA),  mission  completion  time  and  team  cognitive 
overload.  These  measures  are  harder  to  collect  as  well  as  harder  to  evaluate. 
Sometimes  it  is  hard  to  determine  if  mission  failure  was  due  to  the  equipment  or 
to  team  dynamics/characteristics. 

c.  Distributed  performance  measures  are  the  hardest  to  measure.  During  these  types 
of  measurements,  team  members  work  together  toward  a  goal  but  they  are  not  co¬ 
located.  Items  such  as  message  send/retrieval  rate,  dropped  messages,  and 
mission  completion  time  all  can  be  measured  distributively. 

3.8  Step  8  -  Develop  Questionnaires,  Interviews,  Surveys,  and  Data  Collection  tools. 

Valid  and  reliable  data  reflecting  personnel  opinions  and  insights  concerning  Soldier-equipment 
system  performance  capability,  military  utility,  and  Soldier  acceptance  shall  be  obtained  through 
the  use  of  carefully  prepared  and  administered  questionnaires  and  interviews.  These  methods  are 
employed  to  obtain  opinions,  attitudes  and  preferences  of  personnel  who  have  “hands-on” 
experience  with  the  equipment. 

Questionnaires,  interviews,  and  surveys  can  serve  as  additional  data  collection  tools.  They  can 
collect  data  on  variables,  such  as  workload  and  SA,  where  only  partial  instrumentation 
measurement  and  analysis  capabilities  exist.  They  also  serve  to  supplement  and  clarify 
information  derived  from  observations  and  measurements  concerning  identification  of  use 
problems,  magnitude  of  problems,  causal  factors,  and  test  participant  perceptions  of  implications 
and  effects  of  such  problems.  One  place  to  find  guidance  in  the  preparation  of  questionnaires, 
interviews  and  surveys  is  provided  in  DTC  PAM  602-1. 

3.9  Step  9  -  Identify  Test  Participants.  When  performing  operational  tasks,  test 
participants  need  to  be  representative  of  the  user  population  in  terms  of  MOS,  skill  level,  and 
training.  Test  project  personnel  therefore,  shall  review  the  item  documentation  to  determine  the 
following  characteristics  of  the  intended  user  population.  In  addition,  any  specific  training 
programs  necessary  to  operate,  maintain,  or  otherwise  use  the  item  must  be  identified.  The 
treatment  of  the  test  participants  should  follow  AR  70-25:  Use  of  Volunteers  as  Subjects  of 
Research  procedures. 

The  names  of  test  participants  will  not  be  used  in  test  reports,  so  each  participant  should  be 
identified  by  a  random  code.  This  code  needs  to  be  consistent  between  the  various  HSI 
procedures  and  data  forms  (i.e.  questionnaire/  interview  responses  and  comments,  demographics, 
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anthropometry,  tasks  checklists,  speech  intelligibility,  etc.).  The  code  should  be  randomly 
assigned  and  should  be  consistent  with  the  participant’s  code  on  previous  tests. 

If  Soldier  operator,  maintainer,  test  and  evaluation  (SOMTE)  personnel  are  used  as  test 
participants,  consult  DTC  REG  73-6  for  proper  use. 

The  information  listed  below  represents  sample  criteria  for  subject  selection.  The  sample  size 
for  the  test  is  based  on  the  required  confidence  interval  and  can  be  found  using  DTC  TOP  3-1- 
00259  (25  JAN  67):  Confidence  Interval  and  Sample  Size.  The  distribution  of  each  characteristic 
among  the  test  participants  shall  be  similar  to  that  of  the  target  population  within  selection 
constraints.  The  characteristics  to  be  determined  and  recorded  for  all  participants  shall  include 
consideration  of  the  following: 

a.  Gender  -  The  representation  of  males  and  females  in  the  test  sample  should  be 
similar  to  that  of  the  intended  user  population. 

b.  Physical  Dimensions  -  The  ranges  of  heights  and  weight  shall  be  specified,  giving 
due  consideration  to  the  range  of  these  dimensions  expected  of  typical  user 
personnel  when  the  system  is  fielded.  Specifically,  the  range  should  encompass 
the  5th  through  95th  percentile  in  pertinent  dimensions. 

Since  the  percentile  values  are  not  additive  between  different  dimensions,  there  is 
not  a  typical  5th  or  95th  percentile  Soldier.  For  example,  it  is  incorrect  to  assume 
that  the  combination  of  the  5th  percentile  values  will  describe  the  dimensions  of  a 
"5th  percentile  man." 

Determination  must  be  made  of  specific  body  dimensions  of  importance  for  item 
use  (reach,  seated  height,  kneeling,  etc.).  No  person  with  a  special  duty  or  limited 
duty  profile  can  be  pennitted  to  participate  unless  a  task  analysis  reveals  that  the 
restriction  to  his/her  activities  have  no  impact  on  the  tasks  required  in  the  test. 

c.  Sensory  Acuity  -  All  test  participants  should  have  had  a  recent  (within  the  last  12 
months)  test  of  vision  and  audition.  If  vision  or  audition  are  critical  to  the  test 
function  (or  can  be  impaired  during  function),  the  appropriate  test  shall  be  given 
both  immediately  before  and  after  test  operations.  Minimum  standards  should  be 
stated  for  each  of  these  sensory  modalities  depending  upon  an  analysis  of  the 
requirements  of  the  tasks  to  be  performed.  The  inclusion  of  participants  who 
wear  glasses  should  be  considered  if  appropriate  to  the  particular  test. 

d.  MOS  -  The  required  MOS  will  normally  be  specified  in  the  MANPRINT 
Management  Plan,  Target  Audience  Description,  and  potentially  in  the  SEP,  DTP, 
or  other  test  directive.  If  it  is  not,  a  determination  must  be  made  of  the  MOS  and 
whether  the  specified  MOS  must  be  the  test  participant's  primary  MOS  or  whether 
a  Soldier  with  this  specialty  in  a  MOS  is  acceptable.  Alternate  specialties  may  be 
acceptable  substitutes. 

e.  Grade  -  Test  participants  should  represent  the  grades  and  skill  levels  specified  for 
test  item  users  or  as  specified  in  the  ICD/CDD. 
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f.  Item  Specific  Training  -  If  training  is  required  for  use  of  the  test  item,  this 
training  shall  be  provided  to  test  participants  prior  to  data  collection. 

3.10  Step  10  -  Identify  Test  Facility/Instrumentation  Requirements.  Facilities  need  to  be 
properly  equipped  and  instrumentation  needs  to  be  calibrated  to  standards,  as  required. 
Identification  of  facilities/instrumentation  is  dependent  on  the  test  procedures  being  used  and  the 
availability  of  test  assets.  Specific  test  procedures,  explained  in  individual  TOPs,  will  prescribe 
facility/instrumentation  requirements.  Facility/instrumentation  requirements  may  be  found  or 
implied  in  evaluation  planning  documents. 

3.11  Step  11  -  Identify  Test  Controls.  Test  controls  to  be  implemented  in  a  given  HSI- 
related  test  must  be  appropriate  for  that  test.  Every  test  control  is  not  needed  for  every  test  but  is 
required  when  appropriate.  It  is  important  to  control  difference  between  test  events  to  minimize 
the  impact  on  data  variability.  If  the  differences  are  properly  controlled,  the  identified  variability 
(or  biases)  among  test  conditions  will  be  due  to  only  the  test  item. 

3.11.1  Control  of  Test  Participants.  Selection  of  personnel  to  serve  as  operators  and 
maintainers  should  be  based  on  the  characteristics  explained  in  step  9.  The  selection  should  be  a 
representative  sample  of  the  target  user  population.  This  should  minimize  the  cumulative  effect 
of  experience,  motivation  and  skill  sets  on  test  execution. 

3.11.2  Control  of  Test  Facilities/Instrumentation.  In  addition,  the  test 
facilities/instrumentation  in  use  for  a  test  should  remain  constant  during  each  of  the  test  events. 

If  the  facilities/instrumentation  are  changed  during  a  test  this  will  increase  data  variability. 

3.11.3  Control  of  Test  Procedures.  The  use  of  consistent  procedures  among  test  events  is 
necessary  to  decrease  the  data  variability  across  the  test  conditions.  To  accomplish  consistency 
operators  and  maintainers  are  to  perfonn  according  to  standard  procedures  as  indicated  in  the 
appropriate  technical  manuals  or  operator  and  maintenance  instructions  accompanying  the  test 
item.  This  ensures  more  valid  inferences  from  test  results  with  regard  to  representative 
participants. 

Not  only  must  the  test  participants  follow  standard  procedures,  test  personnel  and  administrators 
must  follow  standard  procedures.  Not  only  does  this  decrease  the  data  variability  between  test 
events,  it  allows  for  repeatability  of  data  for  future  follow-up  test  events. 

3.11.4  Control  for  Comparison  of  Competitive  Systems.  When  comparing  competitive  items, 
personnel  operating  the  competitive  items  should  have  comparable  training  (i.e.,  personnel 
should  be  cross-trained  on  the  items  being  compared).  To  minimize  the  effects  of  variability 
(Soldier  and  equipment)  due  to  perfonning  over  a  period  of  time,  whenever  possible,  conduct  the 
comparison  tests  in  the  same  time  frame  under  the  same  environmental  conditions.  To  further 
minimize  the  influence  of  such  sources  of  variability,  if  possible,  repeat  the  test  procedure 
switching  the  person  (or  crew)  from  Item  A  to  Item  B  as  discussed  in  section  3.1 1.5  below. 

3.11.5  Control  of  Assignment  of  Test  Participants  to  Test  Conditions.  When  creating  test 
procedures,  the  tester  may  have  to  perform  several  tests  where  different  environments  or  item 
configurations  are  required.  For  example,  a  universal  workstation  may  have  to  be  tested  in  a 
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static,  mobile,  and  shipboard  configuration.  Ideally  a  large  group  of  test  participants  should  be 
randomly  assigned  to  the  various  test  conditions.  This  will  allow  each  participant  to  be  involved 
in  only  one  event.  This  would  eliminate  practice  and  order  effects  for  the  different  test 
conditions. 

If  test  participants  are  used  in  every  test  condition  individual  differences  will  not  affect  the  data, 
because  their  individual  differences  will  be  the  same  across  all  of  the  test  conditions.  The  term 
individual  differences  refer  to  the  fact  that  on  any  task  certain  test  participants  will  perform 
better  than  others.  This  may  be  due  to  differences  in  sensory  acuity,  strength,  memory  capacity, 
experience  on  similar  tasks  or  whatever.  The  goal  should  be  to  control  this  source  of  variability 
in  performance  so  that  it  does  not  bias  the  test  results.  Whenever  the  data  for  item  A  arise  from 
one  group  of  test  participants  and  the  data  for  item  B  arise  from  another  group,  apparent 
differences  in  performance  between  items  A  and  B  could  actually  be  due  to  differences  in  skill 
between  the  groups  and  the  test  would  be  biased.  So  whenever  possible,  the  same  group  of  test 
participants  should  be  used  in  each  test  condition. 

There  are  cases  where  this  approach  is  not  possible,  where  test  participants  from  one  condition 
are  not  available  for  another  condition,  or  inclusion  of  participants  from  a  previous  event  would 
skew  the  results.  For  example  if  a  test  has  only  ten  targets  available  for  an  evaluation  of  a  target 
detection  system,  the  same  test  participants  cannot  be  used  a  second  time  because  they  would 
know  where  all  of  the  targets  are  located.  If  randomizing  participants  into  test  conditions  is  not 
possible,  then  split  the  test  participants  into  groups  and  use  the  selection  characteristics  from  step 
9  to  achieve  equivalent  (or  as  equivalent  as  possible)  test  participant  groups  using  a  matched 
groups  design. 

3.11.6  Control  of  Order  of  Presentation  of  Test  Conditions.  Given  that  test  participants  will 
serve  under  each  test  condition,  biases  might  be  formed  due  to  the  order  of  presentation.  Since 
each  test  participant  will  perform  tasks  under  each  test  condition,  the  conditions  will  have  to  be 
presented  in  some  order  such  as  A  first,  B  second  and  C  third.  The  problem  here  is  that 
participant’s  perfonnance  will  probably  improve  over  the  course  of  the  test  because  of  practice 
as  the  tasks  are  repeated.  In  the  case  above,  if  performance  under  condition  C  were  better  than 
under  B  and  better  under  B  than  A,  there  would  be  no  way  to  tell  if  this  is  due  to  true  differences 
attributable  to  the  conditions  or  due  to  learning  on  the  part  of  the  participants.  The  solution  is  to 
counter-balance  the  order  of  presentation  by  varying  this  from  one  participant  to  another.  This 
can  be  done  by  randomly  assigning  participants  to  groups  where  there  are  as  many  groups  as 
there  are  test  conditions.  If  four  conditions  are  to  be  tested  and  twenty  participants  are  available 
then  groups  of  five  would  be  used. 

When  assigning  test  participants,  it  is  sometimes  suggested  that  sufficient  practice  should  be 
given  before  data  collection  begins  and  that  perfonnance  has  improved  due  to  practice  as  much 
as  it  is  going  to.  The  learning  curve  will  then  be  at  an  asymptotic  value  or  participants  will  be  at 
the  top  of  the  learning  curve.  Unfortunately,  human  performance  on  skilled  tasks  generally 
continues  to  show  marginal  improvement  after  months  of  daily  practice  and  thousands  of  trials. 
The  preferred  approach  is  to  provide  as  many  practice  trials  as  project  resources  will  permit  and 
then  use  a  counter-balanced  order  of  presentation  of  conditions  as  shown  in  FIGURE  3-1. 
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Some  counter-balanced  orders  of  presentation  of  two,  three,  four,  and  five  test  conditions  are 
shown  in  FIGURE  3-1.  These  types  of  counter-balanced  orders  of  presentation  are  called  Latin 
squares.  For  test  conditions  numbering  greater  than  live,  Latin  squares  can  be  constructed  or 
they  can  be  found  in  an  experimental  design  book.  If  the  Latin  square  is  to  be  constructed  each  of 
the  test  conditions  appears  once  and  only  once  in  each  row  and  in  each  column.  If  any  two  rows 
or  any  two  columns  of  a  Latin  square  are  interchanged,  the  result  is  still  a  Latin  square. 
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FIGURE  3-1:  Recommended  Orders  of  Presentation  of  Test  Conditions 


In  the  two  test  condition  Latin  square  block  of  FIGURE  3-1,  two  test  conditions  A  and  B  are 
shown.  Ten  participants  are  used  in  the  test  and  are  assigned  at  random  to  groups  (1-2)  and 
participant  numbers  (1-10).  If  a  workstation  is  to  be  used  in  a  static  environment  under  condition 
A  and  in  a  mobile  environment  under  condition  B,  then  group  1  (participants  1-5)  will  evaluate 
the  workstation  in  a  static  environment  first  and  will  then  evaluate  it  in  a  mobile  environment. 
Group  2  (participants  6-10)  will  be  in  the  static  environment  first  and  then  in  a  mobile 
environment.  Conditions  A  and  B  will  have  received  an  equal  number  of  trials  by  each  test 
participant  and  under  each  trial  block  ( 1  and  2).  In  principle,  individual  differences  between 
participants  will  cancel  out  and  effects  of  practice  which  are  associated  with  trial  blocks  1  and  2 
will  cancel  out  of  comparisons  between  the  two  conditions. 
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3.12  Step  12  -  Develop  Test  Plan.  The  final  step  before  test  execution  is  to  construct  the  test 
plan  and  obtain  the  necessary  level  of  approval.  Most  HSI  testing  is  a  subtest  within  a  much 
larger  test  and  only  requires  a  test  plan  to  be  inserted  into  a  larger  Detailed  Test  Plan.  Whether 
the  test  plan  is  a  stand-alone  document  or  embedded  within  a  larger  detailed  test  plan,  the  test 
plan  is  still  needed  to  detail  the  necessary  steps  in  completing  the  test.  It  also  serves  as 
documentation  for  the  test. 

3.12.1  Preparation  of  the  HSI  subtest  within  the  Detailed  Test  Plan.  When  the  HSI  test  is  to 
be  a  subtest  of  a  much  larger  test  program,  the  test  plan  is  to  be  written  as  a  part  of  the  Detailed 
Test  plan.  Section  2.0  of  the  Detailed  Test  plan  contains  the  individual  subtests  and  the 
following  three  parts  of  the  subtest  description.  The  HSI  test  plan  shall  be  written  in  accordance 
with  DTC  PAM  73-1.  The  sub  test  plan  shall  follow  the  following  format: 

a.  Objective:  This  shall  be  a  concise  statement  of  the  objective  or  issue  to  be 
addressed  in  the  subtest,  including  the  subtest' s  relationship  with  the  overall  test 
objectives. 

b.  Criteria  and  Data  Analysis:  This  shall  be  a  statement  of  the  criteria  contained  in 
or  referenced  in  the  Evaluation  Planning  Documentation  or  DTC  directive.  It  will 
also  include  a  description  of  the  data  listed  in  the  data  required  section  that  needs 
to  be  reduced  and  analyzed,  and  how  the  comparison  against  the  criterion 
statements  will  be  made.  The  source  of  all  criteria  should  be  clearly  identified 
down  to  the  paragraph  number. 

c.  Test  Procedures  and  Data  Required:  This  shall  detail  both  the  procedures  to  be 
used  in  collecting  the  data  and  test  specific  data  to  be  obtained  during  the  test. 

The  TOP  procedure  should  be  referenced  along  with  a  brief  description  of  the 
procedure.  If  the  procedure  deviates  from  a  TOP  or  in  the  absence  of  a  TOP,  the 
procedure  shall  be  described  in  detail.  This  shall  specify  the  accuracy 
requirements  of  the  data  and  the  numbers  of  samples  or  observations. 

3.12.2  Preparation  of  a  Stand-alone  HSI  Test  Plan.  If  the  HSI  test  is  to  be  stand-alone  and 
not  part  of  a  much  larger  test  program,  then  the  HSI  test  plan  is  to  be  a  self  contained  detailed 
test  plan.  The  format  stills  follows  the  guidance  in  DTC  PAM  73-1.  The  following  is  a  brief 
description  of  the  sections  needed  for  the  HSI  test  plan. 

a.  Section  1 :  This  part  of  the  test  plan  includes  the  system  description,  summary  and 
unique  test  personnel  requirements  sections.  Section  1  gives  the  background  for 
the  test  as  well  as  any  information  that  is  needed  to  explain  the  subtests  in  the  next 
section. 

b.  Section  2:  This  section  contains  the  subtest  test  plan  as  described  above. 

c.  Appendixes:  The  appendixes  give  detail  that  is  needed  for  conducting  the  test. 

The  following  lists  the  six  appendixes  that  are  contained  in  a  Detailed  Test  Plan. 

i.  Test  Criteria 
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ii.  Test  Schedule 

iii.  Informal  Coordination 

iv.  References 

v.  Abbreviations  (optional) 

vi.  Distribution  List 

3.12.3  Review  of  Test  Plans.  Following  preparation  of  the  HSI  Test  Plan,  the  test-project 
personnel  shall  review  the  plan  to  ensure  that  the  factors  which  could  reduce  the  validity  and 
reliability  of  the  results  have  been  eliminated/minimized.  In  particular,  the  test  conditions,  the 
test  design,  and  the  test  controls  are  to  be  reviewed  to  detennine  if  the  planned  test  will  provide 
unbiased  data.  Once  the  individual  test  plans  are  consolidated  together  into  a  Detailed  Test  Plan 
(DTP),  the  DTP  is  provided  to  the  AST  and  the  T&E  WIPT  for  review  and  comment. 

3.12.4  Approval  of  Test  Plan.  After  the  test  plan  review,  the  plan  must  be  submitted  for 
approval.  After  finalization  the  DTP  is  approved  by  the  CG,  DTC  or  his  designee  (usually  the 
test  division  chief). 

If  test  participants  are  used  in  the  test,  a  Human  Use  Committee  (HUC)  decision  may  have  to 
made.  AR  70-25  addresses  conditions  that  require  a  HUC  review.  If  required  the  DTC  Safety 
office  will  require  an  approved  DTP  and  approved  safety  releases  to  be  used  in  the  HUC  review. 

If  the  test  is  conducted  outdoors,  addressing  the  environmental  issues  may  also  be  required  as  a 
part  of  the  approval  process. 

4.  DATA  REQUIREMENTS  AND  ANALYSIS. 

4.1  Data  Required.  Data  to  be  acquired  in  HSI  tests  includes  the  data  that  are  applicable,  as 
appropriate,  to  all  domains  and  will  include  data  that  are  specific  to  the  Design  and  Performance 
tests. 

4.1.1  Data  General  to  All  Types  of  Tests.  There  are  types  of  data  that  are  common  to  most  or 
every  test. 

a.  Demographic  data  on  test  participants;  if  test  participants  are  used 

b.  Descriptions  of  the  test  conditions  and  test  controls 

c.  Description  of  the  test  design,  run  schedule  and  test  conditions  for  each  run 

d.  Descriptions  of  facilities/instrumentation/materials  (including  modifications  from 
non-standard  configurations) 

4.1.2  Data  Specific  to  Equipment,  Technical  &  Design  Characteristics  Tests  Performed. 

The  Equipment,  Technical  and  Design  Characteristics  tests  have  sets  of  data  that  are  common  to 
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these  series  of  tests.  These  sets  of  data  are  taken  from  the  requirements  in  the  system 
documentation  and  are  usually  collected  as  checklists  are  completed.  The  following  is  a  brief  list 
of  the  types  of  data  that  can  be  collected. 

a.  Completed  design  checklists  (e.g.,  Soldier  survivability,  health  hazard) 

b.  Environmental  parameters  (e.g.,  lighting,  noise,  temperature,  humidity) 

c.  Workstation  evaluation  (e.g.,  speech  intelligibility,  visibility,  HCI  evaluation) 

4,1.3  Data  Specific  to  Performance  Tests  Performed.  Performance  Test  data  are  based  on 
instrumentation  readings,  test  participant  feedback  as  well  as  HSI  specialist  observations  and 
analysis.  This  set  of  data  will  help  assesses  the  impact  of  the  Soldier  on  the  machine’s 
performance  and  visa  versa.  The  following  is  an  abbreviated  list  of  data  elements  that  could  be 
collected. 

a.  Error  Rates  and  error  likelihood  estimations 

b.  Task  completion  times 

c.  Situational  Awareness  (Individual  and  team)  and  the  workload  involved  in 
maintaining  SA 

d.  Workload  (cognitive  and  physical)  and  the  impact  on  error  rate  and  task 
performance 

e.  New  equipment  training  effects 

f.  Data  collected  from  interviews/surveys/questionnaires 

g.  Completed  task  checklists 

4.2  Data  Reduction,  Analysis  and  Presentation.  The  data  reduction  and  analysis  should 
follow  prescribed  procedures  from  other  pertinent  TOPs  and  statistical  manuals.  The  degree  to 
which  the  test  item  and  its  subsystems  (maintenance  and  training  packages)  conform  to  HSI 
specifications,  standards  and  requirements  should  be  presented  in  narrative  form.  Instances  of 
nonconformance  should  be  supported  by  relevant  measurements  and  photographic  illustrations. 

4.2.1  Data  Reduction  and  Statistical  Analysis.  The  first  step  in  data  reduction  and  analysis  is 
to  compile  the  raw  data.  If  data  from  several  different  test  sites  and/or  instrument  configurations 
are  used,  then  data  compatibility  should  have  been  addressed  in  the  planning  stages  and  resolved 
here  in  the  data  reduction  phase.  After  the  summary  tables  are  compiled,  the  data  is  ordered  and 
then  checked  for  accuracy.  Outliers  need  to  be  discussed  in  the  analysis,  and  if  warranted  not 
included  in  data  analysis. 

During  many  tests,  multiple  measurements  are  taken  of  a  particular  test  item  characteristic 
because  the  measurements  are  subject  to  random  fluctuation.  Appropriate  descriptive  statistics 
shall  be  calculated  from  the  data.  Applicable  statistics  include  measures  of  central  tendency 
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including  the  mean,  median  and  mode  and  measures  of  variability  including  the  standard 
deviation  and  inter-quartile  range.  Frequency  distribution  analysis  includes  tabulation  of  raw 
frequency  data,  conversion  to  proportions  or  percent  values  and  graphing  using  bar  charts. 

Testers  may  have  to  perfonn  more  advanced  data  analysis.  Advanced  statistical  techniques  such 
as  T-tests,  Chi-square,  F-test,  and  analysis  of  regression  may  be  necessary  based  on  test 
requirements. 

The  material  contained  in  this  TOP  does  not  include  detailed  methods  for  statistical  analysis  of 
data.  Guidance  for  statistical  treatment  of  data  is  contained  in  standard  sources  such  as  National 
Institutes  of  Standards  and  Technology’s  e-Handbook  of  Statistical  Methods  (located  at 
http://www.itl.nist.gov/div898/handbook/)  and  Electronic  Textbook:  StatSoft  (located  at 
http://www.statsoft.com/textbook/stathome.html). 

4.2.2  Design  Assessment.  All  HSI-related  quantitative  measurement  data  (e.g., 
anthropometric,  noise,  illumination,  temperature,  etc.)  should  be  presented  in  tabular  or  graphic 
form  for  direct  comparison  with  the  specified  criteria  and  to  show  the  degree  of  compliance  or 
noncompliance.  The  results  of  observation  checklists  and  questionnaires/interviews  shall  be 
summarized  and  presented  in  tabular  form.  When  an  adequate  sample  size  is  available,  the 
results  should  be  submitted  to  statistical  analyses. 

4.2.3  Performance  Assessment.  The  data  should  also  be  analyzed  to  find  the  cause  of  any 
degraded  Soldier  performance  or  performance  that  fails  to  meet  system  requirements.  By 
examining  the  performance  data  as  well  as  analyzing  questionnaires  and  interviews  the  testers 
should  be  able  to  narrow  the  causes  of  the  performance  deficiency.  The  analysis  of  the  data 
becomes  important  as  currently  there  is  not  instrumentation  that  will  directly  identify 
performance  problems. 

5.  SPECIFIC  TEST  PROCEDURES.  The  selection  of  the  test  procedures  are  dependent 
on  the  item  to  be  tested,  the  relevant  HSI  domains,  the  facilities/instrumentation  available  and  the 
Program  directed  test  data  requirements.  Specific  procedures  need  to  be  consistent  across  the 
test  runs  if  multiple  runs  are  to  take  place.  This  TOP  does  not  describe  specific  test  procedures, 
which  are  covered  in  the  specific  HSI-domain  TOPs. 

5.1  Item  Specific  Test  Procedures.  The  test  item  will  prescribe  which  test  procedures  can 
and  cannot  be  done.  For  instance,  if  the  test  was  for  glove  acceptance  then  a  finger  dexterity  test 
would  be  appropriate,  but  a  run  through  the  manpower/portability  course  would  not  be 
appropriate.  The  test  procedures  conducted  on  previous  models  of  the  same  item  could  also  be 
use  and  they  would  also  serve  as  a  basis  of  comparison  of  one  model  to  the  next. 

5.2  MANPRINT  Domain  Specific  Test  Procedures.  The  MANPRINT  domain  to  be 
assessed  will  prescribe  which  test  procedures  can  be  used.  If  a  Soldier  survivability  assessment 
is  to  be  accomplished  then  a  manpower  estimate  or  a  habitability  assessment  is  not  needed  for 
the  test.  Many  of  the  MANPRINT-domain  specific  TOPs  will  outline  which  procedures  are 
appropriate  for  each  item  and  test  item  function.  For  example  TOP  1-2-610  defines  and 
classifies  the  test  procedures  for  a  human  factors  engineering  domain  test. 
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5.3.  Facility /Instrumentation  Specific  Test  Procedures.  The  use  of  the 

facility/instrumentation  will  prescribe  which  procedure  can  be  used.  For  example,  testers  can 
collect  operator  mental  workload  data  using  a  combination  of  physiological  instrumentation, 
questionnaires  and  surveys.  Some  types  of  instrumentation  might  be  facility  dependent,  while 
others  can  be  used  independently  of  a  facility.  Restrictions  on  money/time  might  also  restrict  the 
available  choice  for  facilities/instrumentation. 

When  a  test  procedure  is  to  be  repeated  on  the  item  or  between  items,  then  it  is  essential  to  use 
the  same  facility/instrumentation  to  achieve  repeatability  of  the  results.  This  is  important  when 
the  tester  is  tasked  to  compare  two  similar  items  (e.g.,  current  production  item  and  a  legacy 
item). 


5.4  Test  &  Evaluation  Master  Plan  Specific  Test  Procedures.  When  the  required  test  is 
part  of  an  acquisition  program,  program  documentation,  such  as  the  TEMP,  will  provide  test 
guidance.  If  the  documentation  does  not  specify  the  test  procedures  to  be  used  then  it  will 
specify  the  data  (and  associated  tolerance  levels)  to  be  collected.  The  data  required  will 
prescribe  the  test  procedures  to  be  used.  Certain  data  elements  can  only  be  produced  by  certain 
test  procedures,  making  the  selection  dependent  on  the  data  needed. 
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APPENDIX  A.  SPECIFIC  HSI  T&E  TOOLS 

Typical  HSI  Test  and  Evaluation  (T&E)  Tools 

Representative  tools  available  to  testers  for  measuring  and  documenting  various  aspects  of 
Soldier  performance  during  T&E  are  shown  in  Table  1.  Note  that  no  one  tool  alone  is  capable  of 
providing  data  to  sufficiently  assess  phenomena  such  as  cognitive  performance  or  SA.  Thus, 
tool  use  must  be  tailored  and  applied  for  test  events,  as  appropriate. 


Table  1.  T&E  Tools 


NASA  Task  Load  Index  (TLX)  [Physical  and  mental  workload] 

Software  Usability  Measurement  Inventory  (SUMI) 

Mission  Awareness  Rating  Scale  (MARS)  [SA  and  SA-related  workload] 

Size,  activity,  location,  unit,  time,  and  equipment  (SALUTE)-based  survey1 

Multiple  Affect  Adjective  Check  List  -  Revised  (MAACL-R)  [Psychological  stress] 

Cognitive  Readiness  Prediction  Tool  (CRPT) 

Adaptive  Character  of  Thought-Rational  (ACT-R) 

IMPRINT 

Command,  Control,  and  Communication  Techniques  for  Reliable  Assessment  of  Concept 
Execution  (C3Trace) 

SART  (Situation  Awareness  Rating  Technique) 

Times  to  complete  and  errors  associated  with  conducting  critical  battle  command  tasks 

Secondary  perfonnance  measures  for  workload 

Situation  Awareness  Global  Assessment  Technique  (SAGAT)  [Primarily  for  use  in 
simulators] 

Demographic  survey  to  include  personal  characteristics,  training,  and  experience  with 
automated  systems 

Six-degree-of-freedom  motion  table 

Test  Incident  Report 

MANPRINT  Problem  Report 

Detailed  design  checklists 

Documented  observations  of  mission  tasks  conducted  in  accordance  with  current  and/or 
emerging  tactics,  techniques,  and  procedures  (TTP) 

Tailored  questionnaires  and  interviews  using  scaled  survey  responses  (e.g.  Likert  scales) 

After  Action  Review  (AAR)  results 

Detailed  task  checklists  documenting  Soldier  behaviors  (e.g.  actions,  verbalizations 
within  a  relevant  context) 

Mission  and  Means  Framework  (MMF) 

Physiological  measurement  instrumentation  (heart  rate  and  core  temperature) 

Methods  to  address  Soldier  exposure  to  whole-body  vibration  [ISO  2631-1] 

Ground  truth  based  on  documented  observations  and  sensor  data 

Human  Computer  Interface  characteristics  based  on  applicable  standards  and  practices 

Measures  associated  with  personnel  occupied  areas:  heating,  cooling,  lighting,  acoustic 
noise  (steady-state  and  impulse),  and  ventilation 

Workspace  accommodation  for  the  range  of  Soldiers  attired  in  appropriate  mission 
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oriented  protective  postures _ 

Speech  intelligibility  measurements  at  crew  positions  inside  and  outside  platforms  (e.g. 
modified  rhyme  test) _ 


1  ARL-HRED  used  this  survey  in  a  summer  2006  FCS  C4ISR  exercise  at  Fort  Dix  to  obtain  SA  data  in 
an  on-the-move  environment. 

2.  At  the  Aberdeen  Test  Center  (ATC)  the  Air/Ground  Testbed  is  a  distributed  suite  of  tools  that  inserts 
the  Soldier,  vehicle,  and/or  other  test  component  into  an  extreme  simulated  battlefield  environment. 
Currently,  this  is  achieved  through  a  six-degree-of- freedom  motion  table  vehicle  dynamics  simulator. 

The  table  provides  laboratory  control,  which  facilitates  repeatability  of  defined  scenarios,  quick  feedback 
from  users,  and  Soldier  in-the-loop  testing  of  various  design  alternatives.  Data  generated  using  the 
motion  table  can  augment  data  provided  using  surrogates  and  prototypes  and  will  support  modeling  and 
simulation  of  Soldier  performance. 

3  MMF  modeling  and  simulation  efforts  have  demonstrated  the  ability  to  measure  the  mission  impact  of 
Soldier  performance.  The  process  can  also  identify  the  criticality  of  platform  and  Soldier  actions  to 
mission  success  with  respect  to  TTP  and  Alternative  Courses  of  Action  (ACOA).  This  capability  may 
provide  insight  into  the  tactics  and  ACOA  to  be  investigated  during  testing. 
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APPENDIX  B.  ACRONYM  LIST 
AEC  -  Army  Evaluation  Center 
AIS  -  automated  information  systems 
AR  -  Department  of  the  Army  Regulation 

ARL-HRED  -  U.S.  Army  Research  Laboratory;  Human  Research  and  Engineering  Department 

AST  -  Army  Test  &  Evaluation  Command  (ATEC)  Systems  Team 

CB  -  Chemical  &  Biological 

CDD  -  Capability  Development  Document 

CG,  DTC  -  Commanding  General,  Developmental  Test  Command 

DA  -  Department  of  the  Army 

DoD  -  Department  of  Defense 

DODD  -  Department  of  Defense  Directive 

DODI  -  Department  of  Defense  Instruction 

DTC  -  U.S.  Army  Developmental  Test  Command 

DTP  -  Detailed  Test  Plan 

EDP  -  Event  Design  Plan 

HCI  -  Human  Computer  Interface 

HE  -  Human  Engineering 

HFE  -  Human  Factors  Engineering 

HHA  -  Health  Hazard  Assessments 

HSI  -  Human  Systems  Integration 

HUC  -  Human  Use  Committee 

ICD  -  Initial  Capabilities  Document 

ILS  -  Integrated  Logistics  Support 
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IMPRINT  -  Improved  Performance  Research  Integration  Tool 

IWARS  -  Infantry  Warrior  Simulation 

M&S  -  Modeling  and  Simulation 

MANPRINT  -  Manpower  and  Personnel  Integration 

MMP  -  MANPRINT  Management  Plan 

MOS  -  Military  Occupation  Specialty 

MOUT  -  Military  Operation  on  Urban  Terrain 

NASA  TLX  -  National  Air  &  Space  Administration  Task  Load  Index 

NBC  -  Nuclear,  Biological  &  Chemical 

PAM  -  Pamphlet 

PM  -  Program  Manager 

POC  -  Point  of  Contact 

SA  -  System  Assessments 

SA  -  Situational  Awareness 

SART  -  Situational  Awareness  Rating  Technique 

SEP  -  System  Evaluation  Plan 

SER  -  System  evaluation  Report 

SME  -  Subject  Matter  Expert 

SOMTE  -  Soldier,  Operator,  Maintainer,  Test  and  Evaluation 

SORD-PT  -  Soldier-Oriented  Research  and  Development  in  Personnel  and  Training 

STD  -  Standard 

T&E  -  Test  and  Evaluation 

T&E  WIPT  -  Test  and  Evaluation  Working  Integrated  Product  Team 
TEMP  -  Test  and  Evaluation  Master  Plan 
TOP  -  Test  Operationd  Procedure 
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Forward  comments,  recommended  changes,  or  any  pertinent  data  which  may  be  of  use  in 
improving  this  publication  to  the  following  address:  Test  Business  Management  Division 
(CSTE-DTC-TM-B),  US  Army  Developmental  Test  Command,  314  Longs  Corner  Road 
Aberdeen  Proving  Ground,  MD  21005-5055.  Technical  information  may  be  obtained  from  the 
preparing  activity:  Transformation  Division,  CSTE-DTC-AT-TT-T;  US  Army  Aberdeen  Test 
Center,  APG,  MD  21005-5059.  Additional  copies  are  available  from  the  Defense  Technical 
Information  Center,  8725  John  J.  Kingman  Rd.,  STE  0944,  Fort  Belvoir,  VA  22060-6218.  This 
document  is  identified  by  the  accession  number  (AD  No.)  printed  on  the  first  page. 


